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Abstract. This study proposes a virtual reality platform for language learners to 
practice a Target Language (TL) and develop cross-cultural competencies through 
interaction with peers or an Al-agent (limited scope), followed by a scheme for 
engaging learners to envision their conversations by disclosing their thoughts, 
reasoning, feelings, and expectations. This platform, Virtual Reality Conversation 
Envisioning (VRCE), enables the learners to fully share a contextual, immersive 
environment, simulated in Virtual Reality (VR), to have free conversations by 
performing role-plays on proposed topics. VRCE is designed to provide first 
person views during conversation and third person views during envisioning so 
that learners can take the role of participants and meta-participants at each phase. 
Learners envision the conversation individually at certain points during the role-play 
review. Their envisioning is then shared with their conversational partner to detect 
the misunderstandings, observe the situation from each other’s perspective, and 
to learn about sociocultural cues that led to the different viewpoints. Participants’ 
envisioning revealed interesting differences in their understanding of a shared 
situation. Findings showed that VRCE is an effective medium to raise learner 
collaboration and develop cross-cultural competencies. 
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4. Introduction 


There has been increasing interest in the use of virtual worlds and simulated games 
for promoting language learning, raising cultural awareness, and developing 
communicative competencies (Deutschmann & Panichi, 2013; Peterson, 2011). 
Such platforms often focus on teaching discrete cultural points to the learners 
(Cheng, Yang, & Andersen, 2017), promote grammar and vocabulary acquisition 
(Johnson & Valente, 2009), and support real-time communication to fulfill a quest 
(Thorne, 2008). However, there is a paucity of research on making a platform 
that focuses on free-form conversation, the meta-analysis of the interactions, and 
knowledge transfer to develop a set of culture general skills. In today’s culturally 
diverse world, the use of a foreign language has become the main means of 
communication among people of various backgrounds. However, when using 
a foreign language to interact with peers from different cultural backgrounds, 
building common ground on-the-fly becomes a challenge. It is known that learners 
bring their background culture to the conversation even when using a foreign 
language (Lado, 1964). This transfer effect can lead to interesting cross-cultural 
communications as well as peculiar misunderstandings that can be investigated to 
develop sociocultural competencies. 


This paper proposes a platform to support real-time conversation between learners 
or with an AI agent (with limited scope) in VR and to envision the conversation 
by including learners and teachers as meta-participants for making a detailed 
contrastive analysis of the interactions. We conducted an experiment to see if the 
framework is effective in developing learners’ cross-cultural competencies and 
smoothening conversations. We analyzed the learners’ interactions and envisioning 
during the experiment from a sociocultural perspective. 


2. VRCE 


VRCE has two modes, role-play and envisioning. It offers access to engage in 
social interaction with peers, native speakers of English, and an Al-agent (role- 
play), and endows learners with deft tools to reflect on their interactions and 
further perform a contrastive analysis of their reflections (envisioning). A series 
of negotiation scenarios such as bargaining, job offer, etc. were designed as a 
basis for learners’ interaction. An AI agent is built to interact with learners by 
showing appropriate verbal and non-verbal reactions to the learner input on the 
given scenarios. The agent’s reaction is taken from real-world scenarios and is 
augmented by the sociocultural notes, cues on the reasoning behind the agents’ 
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behavior, and hints for expected reactions, provided by meta-participants’ analysis. 
During the interaction with the agent, learners could benefit from the envisioning 
notes that clarified why the conversation led to a particular direction. This part 
demonstrates how envisioning can help understanding the conversation better and 
what it takes to envision a conversation as a meta-participant. 


Voice chat was used to support real-time communication and a motion capture 
system was used to track the participants’ gestures and project them into their 
avatars to allow for natural communication. The platform is designed to support 
recording, rewinding and replaying learner collaborations and enables the learners 
to review, revise, or analyze their interactions. Learners can move back and forth 
through the branches of the conversation to revise their decisions or interaction 
(first person view) or later review their conversation as meta-analyzers to interpret 
their actions, provide notes on instances of cultural implications, and envision the 
conversation (third person view). In envisioning mode, VRCE allows the teachers, 
conversation analysts, or other learners (observers) to add indicators on different 
points of the conversation to prompt for participants’ envisioning. The system 
requests explanation using the rubric shown in Figure |. 


Figure 1. WRCE for role-play and envisioning: (a,b) role-play in first person 
view, (c) role-play review in third person view and envisioning rubric, 
(d) envisioning provided by the participant 


The decision on 
promotion 
entirely depends 
\_ on the empolyer. 


ag 


felt so stupid for 

asking about promotion 

when I heard his answer. 
That was too direct 
and unsympathetic, 


| want to explain... 
N 1 Why | said this, and what! actually meant 
El How | felt/what | perceived after hearing my partner 


3 What! expected my partner to do next 
+ What | think my partner expects me to do next 
5 Other things 
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During envisioning, teachers can join the shared space to elicit learner’s explanation 
and to provide them with necessary scaffold. After envisioning, the system allows 
for replaying the conversation augmented by the envisioning notes. This would 
serve as an example scenario with sociocultural cues to the other learners so that 
they can replay this scenario while benefiting from the cultural points provided by 
their peers earlier. 


3. Experimental analysis 


3.1. Participants 


The participants in this study were 34 upper-intermediate learners of English from 
different cultural backgrounds. Participants were carefully grouped into pairs with 
different cultural backgrounds (e.g. Japanese versus French). 


3.2. Procedure 


The participants were placed in different rooms and did not meet each other before 
the experiment. They received a role-play scenario with secret conditions specified 
for each role. The conditions were set to encourage each participant to negotiate 
for achieving the desired outcome. Participants were asked to perform a role-play 
in first person view using VR. Motion capture was used to transfer their gestures 
into their avatars. The teacher observed the interaction in third person view and 
assigned indicators at various intervals to prompt for learners’ explanation in the 
envisioning phase. After the role-play, each participant reviewed the interaction 
individually (third person view) and envisioned the conversation particularly for 
teacher-specified points. Next, learners were informed that their partner went 
through the same envisioning process, and their explanations will be exchanged to 
see if they had different interpretations. Finally, the two participants were asked to 
meet each other, have a short discussion and fill out a questionnaire. 


4. Results 


Learners’ feedback after they exchanged their envisioning was reflected on a 
five point Likert scale questionnaire (Figure 2). Results show that envisioning in 
VRCE helped the learners to detect and resolve misunderstandings by comparing 
viewpoints and learning about the cultural differences. 
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Figure 2. Participants’ feedback on a five point Likert scale questionnaire for 
exchanged envisioning 


Strongly Strongly 
Agree Disagree 
s 4 3 2 1 
| eee | think the points given by my partner during envisioning help me understand the situation better 
41) | think envisioning helped me understand why my partner reacted in a certain way 
5 | think receiving my partner's envisioning helped me see thing from his/her point of view 
1) | think the envisionings were good enough to clarify the situation 
3 | think most of the envisionings were important to help understand the situation 
1 | think envisioning the cultural points in our conversation was very helpful 
9 | think envisioning the mood and feelings during our conversation was very helpful 
eee I think my partner's envisioning helped me understand why our conversation went to a particular path 
1) | think the envisioning showed some of the points that | did not know/realize 
TT | think my envisioning helped my partner to see things from my point of view 
61) | think after exchanged envisioning | found at some points we had different understanding of the situation 
132 | think exchanging envisioning helped me detect the misunderstanding points in our conversation 


This result is supported by the collected data from the interactions and envisioning. 
Figure 3 shows an excerpt of the learners’ interaction and their individual 
envisionings. As can be seen, what counted as an annoying direct question in one’s 
culture and an instance of galling one’s pride, was no more than a simple question 
out of curiosity that can be asked directly in another culture. By comparing the 
envisioning, both participants realized that they had different understandings of 
the same situation. Another interesting point in this excerpt is the negotiation of 
meaning when one participant misuses the word “disposable”, which is noted by 
the peer. 


Figure 3. A sample excerpt of role-play and envisioning 


A [employee]: If the salary is not enough to let me pay my loan, | have to look into other options. 
B [employer]: But do you have any other option? Like, did you send other resumes to other 
companies? 

[A’s Envisioning}: This was a bit degrading by the employer. Yes, of course | do have a second 
option. If | am going to find a job, | wouldn't put all my eggs in one basket. That wasn't nice to hear. 

[B's Envisioning]: if she had already have some interviews and refused them, it would be 
interesting to know why. Maybe the salary was low or maybe she was not interested into that 
company and if she didn't get any interview yet she may realize that she lack time and my proposal 
can have some kind of advantage. 


~ B [employer]: So, would you be disposable next week if | can propose you a higher salary? 
A [employee]: [pause] Could | start next week? 

B [employer]: Hmm. You can start next week, can you? 

<~) A [employee]: Yes, | believe | can. If we get all the paper work everything in order 


_ gg B lemployer}: Well, this is my part. 


5. Discussion and conclusion 


Results suggest that the majority of participants highlighted the effectiveness of 
VRCE in facilitating cross-cultural interactions. Participants noted that contrastive 
analysis of their envisioning significantly assisted them in interpreting their 
interaction accurately, which in turn may lead to smoother future communication 
with their partner. Learners also pointed that reviewing the interaction in the 
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third person view facilitated meta-analysis of the conversation and raised meta- 
linguistic awareness. One main complaint about this system was the lack of facial 
expressions in VR, which can be addressed by using a web-based 3D version in 
future. 


Further analysis of participants’ interactions in VRCE showed rich use of the 
TL both during role-play and envisioning. VRCE can be an effective medium 
to improve learner competencies by inducing activities that elicit TL dialogues. 
These activities involve collaboration, assistance, and co-construction such as 
negotiation of meaning, asking for clarification, resolving misunderstandings, 
and receiving support from more proficient peers, that are conducive to the 
operation of zones of proximal development (Vygotsky, 1978). It can be inferred 
from the observations that with frequent instances of collaborative learning (e.g. 
explaining specific cultural points) using the TL during role-play, envisioning and 
discussion, this framework can promote TL learning. Data showed that learners’ 
interactions involved a high degree of learner input, as they actively engaged in 
the interpretations of the TL use and sociocultural points which promoted critical 
thinking. Finally, the game-like nature of the scenario raised learner motivation 
and the use of our fully immersive environment allowed the participants to get 
into their roles and benefit from situated and experiential learning (Kolb, 2014). 
In this view, VRCE is anticipated to promote fluency and smoother cross-cultural 
interaction in the long-term. 
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